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ABSTRACT 



The Enterprise Florida Jobs and Education Partnership has 
taken actions to resolve issues raised in a previous report. The legislature 
has expanded the partnership's work force development (WFD) -role; assigned it 
a role in implementing the state's welfare- to-work program; and established 
additional incentive funding for WFD program outcomes. The partnership has 
expanded its evaluation efforts to address requirements in recent 
legislation. It has developed both baseline measures necessary to determine 
whether performance outcomes are improving from year to year, as well as 
expected performance levels necessary to evaluate whether WFD programs are 
achieving desired outcomes. The partnership's procedures for identifying and 
eliminating poor performing programs include evaluating the performance of 
local WFD boards and requiring corrective action when appropriate. Local WFD 
boards refer clients to community college and school district job training 
programs. The partnership uses Performance-Based Incentive Funding (PBIF) to 
encourage community colleges and school districts to provide job training 
programs that prepare individuals for occupations that meet market demands 
and provide a living wage. Disadvantaged student outcomes (completion, 
employment rates, wage levels) show substantial increases since the PBIF 
program began. The partnership monitors community college and school district 
PBIF performance to prevent loss of federal funds. ( YLB) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 
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Abstract 

® The Enterprise Florida Jobs and Education 
Partnership (the Partnership) has taken actions 
in response to the issues in our earlier report. 
The Partnership developed baseline measures 
that are necessary to determine if performance 
outcomes are improving from year to year. The 
Partnership also developed expected 
performance levels that are necessary to 
evaluate whether workforce development 
programs are achieving desired outcomes. 

• The Partnership’s procedures for identifying 
and eliminating poor performing programs 
includes evaluating the performance of local 
workforce development boards and requiring 
corrective action when appropriate. Local 
workforce development boards refer clients to 
community college and school district job 
training programs. The Partnership uses BPIF 
(Performance-Based Incentive Funding) to 
encourage community colleges and school 
districts to provide job training programs that 
prepare individuals for occupations that meet 
market demands and provide a living wage. 
The Partnership does not have a mechanism for 
requiring that community colleges and school 
districts discontinue poor performing programs. 
The Partnership has not resolved the issue of 
consistent outcome measures for workforce 
development programs. 

® Disadvantaged student outcomes (completion, 
employment rates, and wage levels) show 
substantial increases since the PBIF program 
began in Fiscal Year 1994-95. The Partnership 
monitors community college and school district 
PBIF performance to prevent loss of federal 
funds. 



O 




Purpose 

In accordance with s. 11.45(7)(f), F.S., this follow-up 
report informs the Legislature of actions taken by the 
Enterprise Florida Jobs and Education Partnership (the 
Partnership) in response to our Report No. 95-16, 
which we issued December 19, 1995. This report 
presents our assessment of the extent to which the Jobs 
and Education Partnership (also known as the 
Workforce Development Board) has addressed the 
findings and recommendations included in our report. 



Background 

Enterprise Florida, Inc, focuses on creating higher- 
paying jobs through technology innovation and capital 
formation. The Legislature established the Partnership 
in 1994 as an affiliate of Enterprise Florida, Inc. The 
Partnership is a 30-member board with a staff of 18 
located in Tallahassee. The purpose of the Partnership 
is to create and maintain a highly skilled workforce 
able to respond to rapidly changing technology and 
market opportunities essential to a competitive Florida 
economy. 

In 1994, the Legislature also created the High 
Skills/High Wage initiative through the Performance- 
Based Incentive Funding (PBIF) program for 
community colleges and school districts that prepare 
students for high-demand, high-wage jobs as identified 
by the Occupational Forecasting Conference 1 . The 
Partnership distributed $19.4 million in PBIF 



The Occupational Forecasting Conference consists of representatives from 
the Department of Education, Department of Labor and Employment 
Security, the Governor's Office of Tourism, Trade, and Economic 
Development, and the Division of Economic and Demographic Research of 
the Joint Legislative Management Committee of the Florida Legislature. 
Florida has several of these consensus-type conferences that are established 
in law. For example, there is an Economic Forecasting Conference and a 
Public School Enrollment Forecasting Conference. The conferees meet 
periodically to develop consensus on projections which then become 
official data from which state agencies, the legislature and other interests 
will work. The Occupational Forecasting Conference conferees consider 
forecasts of job openings, employment, program placements, and earnings 
in reaching consensus on determining high demand, high wage occupations. 
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incentives for performance in Fiscal Years 1994-95 
and 1995-96. 2 

The Partnership charters local workforce development 
boards to coordinate workforce programs in local 
districts throughout the state. In Fiscal Year 1997-98, 
the Board received a $1.2 million appropriation for 
coordination and administration of workforce 
development in Florida. The Legislature appropriated 
$26 million for two workforce development programs 
administer by the Partnership. The Quick Response 
Training Program received $4 million in general 
revenue and the PBIF program received $22 million 
from the following sources: 

• $2 million from community colleges, 

• $4 million from school districts, 

• $6 million from Project Independence, and 

• $10 from federal Job Training Partnership Act 
(JTPA), Title III, funds that Florida receives for 
providing job training serves to workers displaced 
by a permanent closing or substantial layoffs within 
a business. 3 



Prior Findings 

The Partnership had adopted a research design with 
outcome measures for three of its six statutory goals, 
but to more effectively evaluate performance 
outcomes, needed to 

• develop additional outcome measures with 
baselines and expected performance levels for all 
outcome measures, 

• coordinate with the Florida Education and Training 
Placement Information Program (FETPIP) and 
agencies that administer workforce development 
programs to improve its system for evaluating the 
state's workforce development efforts. 

• provide a mechanism for evaluating major 
employment training programs and eliminating 
poor performing programs; and 

• develop consistent performance measures for valid 
comparisons of outcomes across programs. 

When we conducted our analysis in 1995, preliminary 
data indicated the Partnership's PBIF program was 



2 As of April 1998, the Partnership had awarded $8.1 million PBIF 
incentives for Fiscal Year 1996-97 performance. The Partnership has not 
yet completed the calculations of all PBIF incentives awarded for Fiscal 
Year 1 996-97 performance outcomes. 

3 JTPA, Title III, funds allocated for PBIF incentives must be used to serve 
displaced workers, and BPIF incentive claims have not been sufficient to 
use all the Title III funds. To prevent loss of federal funding, the Partnership 
^-itors PBIF incentive payments and reallocates unexpended PBIF Title 

ands to support local JTPA projects that serve displaced workers. 



producing some positive short-term results. Student 
outcomes (completion, employment rate, and salary 
levels) improved for programs that received PBIF 
incentives. 

In Fiscal Year 1994-95, the first year of its operation, 
the PBIF Program failed to expend a portion the JTPA 
Title III funds allocated for performance incentives. 
Consequently, the U.S. Department of Labor reduced 
Florida's JTPA, Title III, funding by $3.8 million in 
1995-96. We recommended the Partnership develop a 
monitoring system to ensure that PBIF funds are used 
for maximum benefit. 



Current Status 

The Legislature and the Jobs and Education 
Partnership have taken actions to address the issues in 
our prior report. The Legislature expanded the 
Partnership's workforce development role and assigned 
the Partnership a role in implementing the state's 
welfare-to-work program. The Legislature established 
additional incentive programs for workforce 
development programs outcomes such as completion 
and job placement. The Partnership expanded its 
evaluation efforts to address requirements in recent 
legislation and issues raised in our report. 

Legislative Actions 

Workforce Development. In the Workforce Florida 
Act of 1996, the Legislature expanded the Jobs and 
Education Partnership's workforce development role. 
The Act assigned the Partnership the role of Florida's 
State Human Resource Investment Council. The 
Council is responsible for coordinating federal Job 
Training Partnership Act (JTPA) programs. The 
Partnership is responsible for granting charters to 
regional workforce development boards with 
membership consistent with federal and state law. 
According to the Act, the regional workforce 
development boards must provide the following 
initiatives: 

• School-to-Work provides career development and 
job preparation skills for public school students. 

• Welfare-to-Work encourages self-sufficiency for 
welfare recipients. In October 1996, the Work and 
Gain Economic Self-sufficiency (WAGES) Act 
program became the primary mechanism for 
implementing this initiative. 

• High Skills/High Wage identifies the availability 
of high wage jobs and provides incentives to state 
job training programs that prepare individuals for 
these jobs. Jobs are identified through the 
Occupational Forecasting. 



• One-Stop Career Centers provide clients with 
information about the full range of services 
available from the state (e.g., unemployment 
benefits, job placement, job training) in a single 
visit. 

The Act requires the Partnership establish measures 
and standards to evaluate workforce development 
strategies. (See Exhibit 1 .) 

Multiple Incentives Now Available. The Legislature 
uses incentive to tie funding to program outcomes to 
encourage programs to produce the desired outcomes. 
The Legislature's first effort to tie funding to workforce 
development program outcomes was through PBIF. 
The PBIF program began providing incentive funds for 
workforce development outcomes in Fiscal Year 
1994-95. In Fiscal Year 1996-97, the Legislature made 
additional incentives funds available through 
Performance-Based Program Budgeting (PE?). Under 
PB 2 , the Legislature appropriated $12 million in 
1996-97 and $12 million in 1997-98 to provide 
incentives to community colleges for workforce 
development outcomes such as program completion 
and job placement. 



The Legislature Created Workforce Development 
Education Fund. The Legislature created a third 
incentive fund in Ch. 97-307, Laws of Florida. The 
Workforce Development Education Fund links 
program funding to performance outcomes. Student 
enrollments determine initial funding levels. 
Community colleges and school districts earn 
additional funds for such as program completion and 
job placement. The Legislature scheduled 
implementation for July 1, 1998, and created a task 
force to identify any barriers to implementation student 
outcomes. 

Better Information Necessary to Link Finding to 
Performance. In the Workforce Development 
Implementation Act of 1998, the Legislature found that 
the performance-based funding required by Ch. 9707, 
Laws of Florida, cannot be implemented using current 
information available on workforce development 
program outcomes. The 1998 Act requires the 
Department of Education, community colleges, and 
school districts to develop a Workforce Development 
Information System during Fiscal Year 1998-99. The 
act establishes July 1, 1999, as the revised 

implementation date for allocating funds for workforce 
programs through the Workforce Development 
Education Fund. 



Exhibit 1 

The Partnership Is Developing Tier 1 and Tier 2 Performance Measures 



As Required by the Workforce Development Act 


Tier 




PeHonnarice'Measures • 


Tier 1 

Systemwide Outcomes 


• Employment in occupations demonstrating continued growth in wages; 

• Continued Employment after 3, 6, 12, and 24 months; 

• Reduction in and elimination of public assistance reliance; 

• Employer satisfaction; and 

• Positive return on investment of public resources. 


Tier 2 
Impact of 
Major Initiatives 


A set of standards and measures for each of the four initiatives (School-to-Work, Welfare-to-WorJ , 
High Skills/High Wage, and One-Stop Career Centers) shall include: 

1. Job placement rate, 

2. Cost per entered employment, 

3. Earnings at placement, 

4. Employment retention, and 

5. Entered employment rate. 

These initiatives may add measures as follows: 

• School-to-Work 

Number of students enrolling and completing work-based programs. 

• Weifare-to-Work 

1. Reduction in and elimination of public assistance reliance, and 

2. Savings resulting from reduced reliance on public assistance. 

• One-Stop Career Centers 

1 . Placement at minimum wage 

2. Placement at wage level established by Occupational Forecasting Conference 

3. Placement at wage level above that established by the Conference 


Tier 3 

Operational Measures 


May be developed by implementing agencies and may be specific to federal requirements and should 
reflect regional differences such as job availability, unemployment rate, average wages, and availal 
employment population. 



Source: Section 288.9620, F.S. 
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Jobs and Education Partnership Actions 

As required by the Workforce Florida Act of 1996, the 
Partnership developed performance measures for 
evaluating the state's workforce development efforts. 
The Partnership's 1997 performance measures include 
baselines and expected performance levels for 
performance outcomes as recommended in our report. 

The Partnership's procedures for identifying and 
eliminating poor performing programs includes 
evaluating annual performance of regional workforce 
development boards, issuing annual performance 
reports, and requesting corrective action based on 
regional boards' annual performance. The Partnership 
relies upon PBIF to encourage boards to refer clients to 
programs with good performance and discourage 
referrals to poor performing programs, such as 
cosmetology. 

Performance outcomes such as program completion are 
still defined differently among workforce development 
programs. Consequently, valid comparison of program 
performance can only be made within similar 
programs. The Partnership is working with workforce 
development agencies to develop consistency in 
defining program outcomes. 

The Partnership's PBIF outcome data shows positive 
results. Completion and job placement rates increased 
substantially for designated students since the PBIF 
program began in Fiscal Year 1993-94. (See 
Exhibit 2.) 

The state has not lost any additional federal funds due 
unexpended PBIF funds. The Partnership works with 
the Department of Labor and Employment Security 
and the Office of Tourism, Trade and Economic 
Development to update federal JTPA expenditure 
projections throughout the year. The Partnership 
makes adjustments to prevent reductions in the state's 
JTPA allocations. In Fiscal Year 1996-97, the 
Partnership distributed $1.7 million in unexpended 
funds to regional workforce development boards to 
assist with WAGES efforts. 



Exhibit 2 



PBIF Outcomes 1 



Performance 


Baseline 


Year 1 


Year 2 


Change 


Category' 


1992-93 


1994-95 


1995-96. 


1992-95 


f 

All Students 


Completions 


17,472 


18,709 


24,256 


39% 


Completers Placed 


10,213 


14,473 


16,851 


65% 


Leavers Placed 2 


4,963 


13,732 


14,796 




Design a ted Pop it lot ion s* 


Enrollments 


25,587 


42,138 


48,930 


91% 


Completions 


4,886 


6,136 


8,089 


66% 


Completers Placed 


2,118 


4,223 


5,077 


140% 


Leavers Placed 


1,223 


... 4,736 


j 3,089 


153% 



'Outcomes are not yet available for Fiscal Year 1 996-97. 

^Leavers are individuals who leave structured training programs prior to 
completion and subsequently gain employment in the high wage 
occupations identified through the Occupational Forecasting Conference, 
^Designated populations are economically disadvantaged individuals, 
individuals with disabilities, the limited English proficient, and workers 
who have been dislocated from their jobs 
Source: Enterprise Florida Workforce Development Board 

Observation 

The Legislature may want to consider the impact of 
providing multiple incentive funding programs for 
workforce development programs. Multiple incentives 
for the similar outcomes may create conflicting goals 
for community colleges and school districts. The PBIF 
program rewards job-training programs for preparing 
students for high demand occupations identified in the 
Occupational Forecasting Conference. Pl3 funding 
provides incentives for all workforce development 
outcomes regardless of whether the training is in a high 
demand occupation. The Workforce Development 
Fund will provide incentives similar to those provided 
by the PB 2 funding. Consequently, job training 
programs will have to choose between conflicting 
goals of multiple incentive programs in providing 
workforce development programs. Job training 
programs also have additional reporting requirements 
and more complexity in the budgeting process with 
multiple incentive programs 
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OPPAGA provides objective, independent, professional analyses of state policies and services to assist the Florida 
Legislature in decision-making, to ensure government accountability, and to recommend the best use of public resources. 
This project was conducted in accordance with applicable evaluation standards. Copies of this report in print or alternate 
accessible format may be obtained by telephone (850/488-0021 or 800/531-2477), by FAX (850/4873804), in person 
(Claude Pepper Building, Room 312, 111 W. Madison St.), or by mail (OPPAGA Report Production, P.O. Box 1735, 
Tallahassee, FL 32302). 
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